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This  project  considers  a  class  of  general  quadratic  optimization  problems  involving  both 
integer  and  continuous  variables.  These  problems  are  strongly  motivated  by  applications 
in  optimal  and  dynamic  resource  management,  cardinality  constrained  quadratic 
programs,  and  the  matrix  completion  problems  with  non-convex  regularity.  The  project 
addresses  a  fundamental  question  how  to  efficiently  solve  these  problems,  such  as  to  find 
a  provably  high  quality  approximate  solution  or  to  fast  find  a  local  solution  with  probable 
structure.  Given  the  non-convex  nature  of  these  problems,  two  relaxation  approaches  are 
considered:  one  is  based  on  convex  (semidefmite)  relaxation  (CVR),  while  the  other  is 
based  on  quasi-convex  relaxations  (QCR).  In  contrast  to  the  classical  mixed  integer 
nonlinear  programming  approaches,  no  convexity  is  assumed  for  sub-problems  when 
some  integer  variables  are  fixed.  For  the  cardinality  constrained  QP  problems,  a  QCR 
approach  is  proposed  based  on  non-convex  regularization.  Compared  to  the  known  LI 
relaxation,  this  relaxation  gives  better  performance  in  practice. 

In  this  project,  we  like  to  do  1)  Theoretical  analysis  of  relaxations  SDR  and  QCR,  and  2) 
Develop  efficient  algorithms  for  solving  large-scale  relaxation  problems,  either  finding  a 
global  or  local  solution. 

Below  are  what  we  have  achieved  so  far. 

I.  Convex  Relaxation  (CVS)  Models  and  Analyses 

“A  Dynamic  Near-Optimal  Algorithm  for  Online  Linear  Programming”  (Agrawal, 
Wang  and  Ye),  Operations  Research,  62(4)  (2014)  876  -  890. 

This  paper  considers  a  typical  online  0-1  integer  resource-allocation  linear  program  -  a 
natural  optimization  model  that  formulates  many  online  resource  allocation  and  revenue 
management  problems  is  the  online  linear  program  (0-1 -ILP)  in  which  the  constraint 
matrix  is  revealed  column  by  column  along  with  the  corresponding  objective  coefficient. 
In  such  a  model,  a  0  or  1  decision  variable  has  to  be  set  each  time  a  column  is  revealed 
without  observing  the  future  inputs  and  the  goal  is  to  maximize  the  overall  objective 
function.  In  this  paper,  we  provide  a  near-optimal  algorithm  for  this  general  class  of 
online  problems  under  the  assumption  of  random  order  of  arrival  and  some  mild 
conditions  on  the  size  of  the  ILP  right-hand-side  input.  Specifically,  our  learning-based 
algorithm  works  by  dynamically  updating  a  threshold  price  vector  of  the  LP  relaxation  at 
geometric  time  intervals,  where  the  dual  prices  learned  from  the  revealed  columns  in  the 
previous  period  are  used  to  determine  the  sequential  decisions  in  the  current  period.  Due 
to  the  feature  of  dynamic  learning,  the  competitiveness  of  our  algorithm  improves  over 
the  past  study  of  the  same  problem.  We  also  present  a  worst-case  example  showing  that 
the  performance  of  our  algorithm  is  near-optimal.  This  paper  is  already  cited  74  times 
according  to  the  Google  Scholar. 
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“Analytical  results  and  efficient  algorithm  for  optimal  portfolio  deleveraging  with 
market  impact,”  (Jingnan  Chen,  Liming  Feng,  Jiming  Peng,  Yinyu  Ye),  Operations 
Research,  62(1)  (2014)  195-206. 

In  this  paper  we  consider  an  optimal  portfolio  deleveraging  problem,  where  the  objective 
is  to  meet  specified  debt/equity  requirements  at  the  minimal  execution  cost.  Permanent 
and  temporary  price  impact  is  taken  into  account.  With  no  restrictions  on  the  relative 
magnitudes  of  permanent  and  temporary  price  impact,  the  optimal  deleveraging  problem 
reduces  to  a  nonconvex  quadratic  program  with  quadratic  and  box  constraints.  Analytical 
results  on  the  optimal  deleveraging  strategy  are  obtained.  They  provide  guidance  on  how 
we  liquidate  a  portfolio  according  to  endogenous  and  exogenous  factors.  A  Lagrangian 
relaxation  method  is  proposed  to  solve  the  nonconvex  quadratic  program  numerically.  By 
studying  the  breakpoints  of  the  Lagrangian  problem,  we  obtain  conditions  under  which 
the  Lagrangian  method  returns  an  optimal  solution  of  the  deleveraging  problem.  When 
the  Lagrangian  algorithm  returns  a  suboptimal  approximation,  we  present  upper  bounds 
on  the  loss  in  equity  caused  by  using  such  an  approximation.  This  piece  of  work  won  the 
Morgan  Stanley  2012  Prize  for  Excellence  in  Financial  Markets,  First  runner-up. 

“Space  tensor  conic  programming,”  (L  Qi  and  Y  Ye),  Computational  Optimization  and 
Applications,  6(26)  (2013)  1-13. 

Space  tensors  appear  in  physics  and  mechanics,  and  they  are  real  physical  entities. 
Mathematically,  they  are  tensors  in  the  three-dimensional  Euclidean  space.  In  the 
research  of  diffusion  magnetic  resonance  imaging,  convex  optimization  problems  are 
formed  where  higher  order  positive  semi-definite  space  tensors  are  involved.  In  this  short 
paper,  we  investigate  these  problems  from  the  viewpoint  of  conic  linear  programming 
(CLP).  We  characterize  the  dual  cone  of  the  positive  semi-definite  space  tensor  cone,  and 
study  the  CLP  formulation  and  the  duality  of  the  positive  semi-definite  space  tensor  conic 
programming  problem. 

II.  Quasi-Convex  Relaxation  (QCR)  Models  and  Analyses 

“Complexity  Analysis  of  Interior  Point  Algorithms  for  Non-Lipschitz  and 
Nonconvex  Minimization,”  (W.  Bian,  X.  Chen,  and  Ye),  Math  Programming,  149  (2015) 
301-327. 

In  this  paper  we  propose  a  first  order  interior  point  algorithm  for  a  class  of  non-Lipschitz 
and  nonconvex  minimization  problems  with  box  constraints,  which  arise  from  emerging 
applications  in  variable  selection  and  regularized  optimization.  The  objective  functions  of 
these  problems  are  continuously  differentiable  typically  at  interior  points  of  the  feasible 
set.  Our  first  order  algorithm  is  easy  to  implement  and  the  objective  function  value  is 
reduced  monotonically  along  the  iteration  points.  We  show  that  the  worst-case  iteration 
complexity  for  finding  an  e  scaled  first  order  stationary  point  is  0(e-2).  Furthermore,  we 
develop  a  second  order  interior  point  algorithm  using  the  Hessian  matrix,  and  solve  a 
quadratic  program  with  a  ball  constraint  at  each  iteration.  Although  the  second  order 
interior  point  algorithm  costs  more  computational  time  than  that  of  the  first-order 
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algorithm  in  each  iteration,  its  worst-case  iteration  complexity  for  finding  an  e  scaled 
second  order  stationary  point  is  reduced  to  0(e-3/2).  Note  that  an  e  scaled  second  order 
stationary  point  must  also  be  an  c  scaled  first  order  stationary  point.  These  results  are  the 
state  of  art  in  complexity  analysis  of  non-convex  optimization. 

“Complexity  of  Unconstrained  L2-Lp  Minimization,”  (Chen,  Ge,  Wang,  Ye),  Math 
Programming,  143  (1-2)  (2014)  371-383. 

This  paper  resolved  an  important  open  question  in  cardinality  constrained  quadratic 
optimization  and  regression.  Here  we  consider  the  unconstrained  Lq-Lp  minimization: 
find  a  minimizer  of  //  Ax-b  //  qAq+k //  x //  pAp  for  given  A  £  Rmxn,  b  £  Rm  and 
parameters  A>0,  p£  [0,1)  and  q^l.  This  problem  has  been  studied  extensively  in  many 
areas.  Especially,  for  the  case  when  q=2,  this  problem  is  known  as  the  L2-Lp 
minimization  problem  and  has  found  its  applications  in  variable  selection  problems  and 
sparse  least  squares  fitting  for  high  dimensional  data.  Theoretical  results  show  that  the 
minimizers  of  the  Lq-Lp  problem  have  various  attractive  features  due  to  the  concavity 
and  non-Lipschitzian  property  of  the  regularization  function  //  •  //  pAp.  In  this  paper,  we 
show  that  the  Lq-Lp  minimization  problem  is  strongly  NP-hard  for  any  p  £  [0,1)  and  q-S 
1 ,  including  its  smoothed  version.  On  the  other  hand,  we  show  that,  by  choosing 
parameters  (p,k)  carefully,  a  minimizer,  global  or  local,  will  have  certain  desired  sparsity. 
We  believe  that  these  results  provide  new  theoretical  insights  to  the  studies  and 
applications  of  the  concave  regularized  optimization  problems.  The  above  two  papers 
have  been  cited  71  times 

“Simultaneous  Beam  Sampling  and  Aperture  Shape  Optimization  for  Station 
Parameter  Optimized  Radiation  Therapy  (SPORT)”  (M  Zarepisheh,  Y  Ye,  S  Boyd,  R 
Li,  L  Xing),  Medical  Physics  41(6)  (2014)  292-292. 

Station  parameter  optimized  radiation  therapy  (SPORT)  was  recently  proposed  to  fully 
utilize  the  technical  capability  of  emerging  digital  linear  accelerators,  in  which  the  station 
parameters  of  a  delivery  system,  such  as  aperture  shape  and  weight,  couch  position/angle, 
gantry/collimator  angle,  can  be  optimized  simultaneously.  SPORT  promises  to  deliver 
remarkable  radiation  dose  distributions  in  an  efficient  manner,  yet  there  exists  no 
optimization  algorithm  for  its  implementation.  The  purpose  of  this  work  is  to  develop  an 
algorithm  to  simultaneously  optimize  the  beam  sampling  and  aperture  shapes. 

In  this  paper  we  build  a  mixed  integer  quadratic  programming  (MIQP)  model  with  the 
fundamental  station  point  parameters  as  the  decision  variables.  To  solve  the  resulting 
large-scale  optimization  problem,  the  authors  devise  an  effective  algorithm  by  integrating 
three  advanced  optimization  techniques:  column  generation,  subgradient  method,  and 
pattern  search.  Column  generation  adds  the  most  beneficial  stations  sequentially  until  the 
plan  quality  improvement  saturates  and  provides  a  good  starting  point  for  the  subsequent 
optimization.  It  also  adds  the  new  stations  during  the  algorithm  if  beneficial.  For  each 
update  resulted  from  column  generation,  the  subgradient  method  improves  the  selected 
stations  locally  by  reshaping  the  apertures  and  updating  the  beam  angles  toward  a  descent 
subgradient  direction.  The  algorithm  continues  to  improve  the  selected  stations  locally 
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and  globally  by  a  pattern  search  algorithm  to  explore  the  part  of  search  space  not 
reachable  by  the  subgradient  method.  By  combining  these  three  techniques  together,  all 
plausible  combinations  of  station  parameters  are  searched  efficiently  to  yield  the  optimal 
solution. 

A  SPORT  optimization  framework  with  seamlessly  integration  of  three  complementary 
algorithms,  column  generation,  subgradient  method,  and  pattern  search,  was  established. 
The  proposed  technique  was  applied  to  two  previously  treated  clinical  cases:  a  head  and 
neck  and  a  prostate  case.  It  significantly  improved  the  target  conformality  and  at  the  same 
time  critical  structure  sparing  compared  with  conventional  intensity  modulated  radiation 
therapy  (IMRT).  In  the  head  and  neck  case,  for  example,  the  average  PTV  coverage 
D99%  for  two  PTVs,  cord  and  brainstem  max  doses,  and  right  parotid  gland  mean  dose 
were  improved,  respectively,  by  about  7%,  37%,  12%,  and  16%. 

“Folded  Concave  Penalized  Sparse  Linear  Regression:  Sparsity,  Statistical 
Performance,  and  Algorithmic  Theory  for  Local  Solutions,”  (H.  Liu,  T.  Yao,  R.  Li,  Y. 
Ye)  manuscript,  2nd  revision  in  Math  Programming  (2016). 

This  paper  concerns  the  folded  concave  penalized  sparse  linear  regression  (FCPSLR) 
problem,  which  is  an  alternative  sparse  recovery  method  than  Lasso.  Although  FCPSLR 
yields  desirable  recovery  performance  when  solved  globally,  computing  a  global  solution 
is  NP-complete.  Despite  some  existing  statistical  performance  analyses  on  local 
minimizers  or  on  specific  FCPSLR-based  learning  algorithms,  it  still  remains  an  open 
question  whether  local  solutions  that  are  known  to  admit  fully  polynomial-time 
approximation  schemes  (FPTAS)  may  already  be  sufficient  to  ensure  the  statistical 
performance,  and  whether  that  statistical  performance  can  be  non-contingent  on  the 
specific  designs  of  computing  procedures.  Seeking  to  address  this  question,  this  paper 
presents  the  following  three-fold  results:  (i)  Any  local  solution  (stationary  point)  is  a 
sparse  estimator,  under  some  conditions  on  the  parameters  of  the  folded  concave 
penalties,  (ii)  Perhaps  more  importantly,  any  local  solution  satisfying  a  significant 
subspace  second-order  necessary  condition  (S30NC),  which  is  weaker  than  the  second- 
order  KKT  condition,  yields  a  bounded  error  in  approximating  the  true  parameter  with 
high  probability.  In  addition,  if  the  minimal  signal  strength  is  sufficient,  the  S30NC 
solution  likely  recovers  the  oracle  solution,  (iii)  We  apply  (ii)  to  the  special  case  of 
FCPSLR  with  minimax  concave  penalty  (MCP)  and  show  that  under  the  restricted 
eigenvalue  condition,  any  S30NC  solution  with  a  better  objective  value  than  the  Lasso 
solution  entails  the  strong  oracle  property.  We  also  show  that  to  guarantee  the  S30NC 
admits  the  FPTAS  algorithm. 

III.  Fast  Algorithm  Development 

“The  simplex  method  is  strongly  polynomial  for  deterministic  Markov  decision 

processes,”  (Ian  Post  and  Ye),  Math  of  Operations  Research,  40  (4)  (2015)  859-868. 

The  simplex  method  continues  to  be  the  major  method  for  integer  linear  programming, 
but  its  complexity  analyses  are  most  negative.  In  this  paper  we  finally  prove  that  the 
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simplex  method  with  the  highest  gain/most-negative-reduced  cost  pivoting  rule 
converges  in  strongly  polynomial  time  for  deterministic  Markov  decision  processes 
(MDPs)  regardless  of  the  discount  factor.  For  a  deterministic  MDP  with  n  states  and  m 
actions,  we  prove  the  simplex  method  runs  in  0(nA3  mA2  logA2  n)  iterations  if  the 
discount  factor  is  uniform  and  0(nA5  mA3  logA2  n)  iterations  if  each  action  has  a  distinct 
discount  factor.  Previously  the  simplex  method  was  known  to  run  in  polynomial  time 
only  for  discounted  MDPs  where  the  discount  was  bounded  away  from  1 .  This  piece  of 
work  won  the  2013  Second  Prize  of  INFORMS  Nicholson  Student  Paper 
Competition. 

“The  Direct  Extension  of  ADMM  for  Multi-block  Convex  Minimization  Problems  is 
Not  Necessarily  Convergent,”  (C.  Chen,  B.  He,  Y.  Ye,  X.  Yuan),  Math  Programming, 
155(1-2)  (2016)  57-79. 

The  alternating  direction  method  of  multipliers  (ADMM)  is  now  widely  used  in  many 
fields,  and  its  convergence  was  proved  when  two  blocks  of  variables  are  alternatively 
updated.  It  is  strongly  desirable  and  practically  valuable  to  extend  the  ADMM  directly  to 
the  case  of  a  multi-block  convex  minimization  problem  where  its  objective  function  is  the 
sum  of  more  than  two  separable  convex  functions.  However,  the  convergence  of  this 
extension  has  been  missing  for  a  long  time — neither  an  affirmative  convergence  proof 
nor  an  example  showing  its  divergence  is  known  in  the  literature.  In  this  paper  we  give  a 
negative  answer  to  this  long-standing  open  question:  The  direct  extension  of  ADMM  is 
not  necessarily  convergent.  We  also  present  a  sufficient  condition  to  ensure  the 
convergence  of  the  direct  extension  of  ADMM,  and  give  an  example  to  show  its 
divergence.  This  paper  is  already  cited  147  times  according  to  the  Google  Scholar. 

“On  the  Expected  Convergence  of  Randomly  Permuted  ADMM,”  (Sun,  LUO  and 
Ye),  in  OPT2015  (Optimization@NIPS  for  Machine  Learning),  2015. 

The  alternating  direction  method  of  multipliers  (ADMM)  is  now  widely  used  in  many 
fields,  and  its  convergence  was  proved  when  two  blocks  of  variables  are  alternately 
updated.  It  is  computationally  beneficial  to  extend  the  ADMM  directly  to  the  case  of  a 
multi-block  convex  minimization  problem.  Unfortunately,  such  an  extension  fails  to 
converge  even  when  solving  a  simple  square  system  of  linear  equations.  In  this  paper, 
however,  we  prove  that,  if  in  each  step  one  randomly  and  independently  permutes  the 
updating  order  of  any  given  number  of  blocks  followed  by  the  regular  multiplier  update, 
the  method  will  converge  in  expectation  for  solving  the  square  system  of  linear  equations. 
Our  result  indicates  that  for  ADMM  the  cyclic  update  rule  and  the  random  permutation 
update  rule  are  fundamentally  different.  To  the  best  of  our  knowledge,  this  is  the  first 
example  that  such  a  difference  is  rigorously  established  for  a  specific  optimization 
algorithm,  although  the  random  permutation  rule  has  been  reported  to  be  experimentally 
better  than  the  cyclic  rule  in  many  large-scale  optimization  problems.  Our  analysis 
technique  of  random  permutation  might  be  useful  in  other  contexts. 
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“ALGORITHMS  FOR  UNSYMMETRIC  CONE  OPTIMIZATION  AND  AN 
IMPLEMENTATION  FOR  PROBLEMS  WITH  THE  EXPONENTIAL  CONE,” 
Ph.D.  Thesis,  Santiago  Akle  Serrano,  Stanford,  2015.  The  thesis  was  founded  by  the 
project,  and  it  produced  a  fast  solver  for  un-symmetric  conic  optimization  solver  included 
in  an  open  source  code  http://web.stanford.edu/~boyd/papers/ecos.html 
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